The 1 Algorithm

Appropriate bias is widely viewed as the key to efficient learning and generalization. I present a new algorithm, the Incremental Delta-Bar-Delta (IDBD) algorithm, for the learning of appropriate biases based on previous learning experience. The IDBD algorithm is developed for the case of a simple, linear learning system-the LMS or delta rule with a separate learning-rate parameter for each input. The IDBD algorithm adjusts the learning-rate parameters, which are an important form of bias for this system. Because bias in this approach is adapted based on previous learning experience, the appropriate testbeds are drifting or non-stationary learning tasks. For particular tasks of this type, I show that the IDBD algorithm performs better than ordinary LMS and in fact finds the optimal learning rates. The IDBD algorithm extends and improves over prior work by Jacobs and by me in that it is fully incremental and has only a single free parameter. This paper also extends previous work by presenting a derivation of the IDBD algorithm as gradient descent in the space of learning-rate parameters. Finally, I offer a novel interpretation of the IDBD algorithm as an incremental form of hold-one-out cross validation. People can learn very rapidly and generalize extremely accurately. Information theoretic arguments suggest that their inferences are too rapid to be justified by the data that they have immediately available to them. People can learn as fast as they do only because they bring to the learning situation a set of appropriate biases which direct them to prefer certain hypotheses over others. To approach human performance, ma chine learning systems will also need an appropriate set of biases. Where are these to come from? Although this problem is well known, there are few general answers. The field of pattern recognition has long known about the importance of feature selection, and the importance of representations is a recurring theme in AI. But in both of these cases the focus has always been on designing in a good bias, not on acquiring one automatically. This has resulted in an accumulation of specialized and non-extensible techniques. Is there an alternative? If bias is what a learner brings to a learning problem, then how could the learner itself generate an appropriate bias? The only way is to generate the bias from previous learning experience (e.g., Rendell, Seshu, & Tcheng 1987). And this is possible only if the learner encounters a series of different problems requiring the same or similar biases. I[ believe that is a correct characterization of the learning task facing people and real-world learning machines. In this paper, I present a new algorithm for learning appropriate biases for a linear learning system based on previous learning experience. The new algorithm is an extension of the Delta-Bar-Delta algorithm (Jacobs 1988; Sutton 1982; Barto & Sutton 1981; Kesten 1958) such that it is applicable to incremental tasks-supervised learning tasks in which the examples are processed one by one and then discarded. Accordingly, I call the new algorithm the Incremental Delta-Bar-Delta (IDBD) algorithm. The IDBD algorithm can be used to accelerate learning even on single problems, and that is the primary way in which its predecessors have been justified (e.g., Jacobs 1988; Silva & Almeida 1990; Lee & Lippmann 1990; Sutton 1986), but its greatest significance I believe is for nonstationary tasks or for sequences of related tasks, and it is on the former that I test it here.